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Abstract 

This note corrects a discrepancy between the semantics and the algorithm of the multiple 
until operator of CSL, like in Pr>o. 0025(0 ^[1,2] b t/[3,4] c), of the article: Model-checking 
continuous-time Markov chains by Aziz, Sanwal, Singhal and Brayton, TOCL 1(1), July 
2000, pp. 162-170. 

1 Introduction 

The widely cited article [1] defines continuous stochastic logic (CSL), a logic to reason about 
continuous-time Markov chains, with a multiple until operator to write formulas (with atomic 
propositions a, 6, and c) like: 

a E/ [1)2 ] b U [3A] c . 

The semantics given in the article is: 

A path 7r satisfies fx U [aiM] f 2 U [a2M] ■ ■ ■ U [an _ ubii _ l] f n "if and only if there exist 
real numbers t\, . . . , t n -x such that for each integer in [1, n) we have (a,; < t i < bi) A 
(W € [ti-ijti])^^)" satisfies /i"), where t-\ is defined to be for notational conve- 
nience." 

This definition uses the undefined variables t, to, i, a n , b n , and t n (while it defines the unused 
variable t-i), and it seems to require that 7r(tj) satisfy fi A fi+i- Obviously, the authors meant 
something like: 

A path 7T satisfies fx U[ aitbl ] f 2 ^[a 2 ,6 2 ] ■•• U[ an _ uK _ 1 ] f n if and only if there exist 
real numbers < tx < t 2 < ■ ■ ■ < t n -i such that for each integer i in [l,n — 1] we 
have (ai < ti < bi) A (Vi' € [U-x, ^))( 7r (^') satisfies fi), where t is defined to be for 
notational convenience, and additionally 7r(£„_i) satisfies /„. 

However, the implementation, i. e. the algorithm that estimates the probability of this until oper- 
ator, uses another semantics implicitly, namely the following: 

A path 7r satisfies fx ^[01,61] f2 C^[a 2 ,b 2 ] ■ • • ^[a„_ 1 ,6„_ 1 ] fn if and only if for each integer 
i in [1, n— 1] we have (Vi' € [6,_i, a»] )(ir(t') satisfies fi) A(Vt' € (ai,bi) )(7r(t') satisfies 
fi V fi+x), where &o is defined to be for notational convenience, and additionally 
7r(& n _i) satisfies /„. 

The implementation allows to switch back and forth between states satisfying fi A->/i+i and states 
satisfying ->fi A fi+x, and it requires to stay in a / n -state longer than the semantics. 

The present article exhibits the error and shows how it can be corrected. In the remainder of the 
article, we will assume that the intervals do not overlap, i. e., that bi < a,i + x for all i = 1,2, ... , ri— 2. 
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Figure 1: Example Markov chain 



2 Example 

For example, consider the Markov chain drawn in Figure [T] The probability that a path satisfies 
the formula g = a U r 1;2 ] ^[3,4] c can be calculated as the product of a few Poisson probabilities: 

Pr(0 transitions during time [0, 1)) • Pr(l transition during time [1, 2]) • |- 

• Pr(0 transitions during time (2,3)) • Pr(> transition(s) during time [3,4]) = 
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However, pQ's algorithm does the following calculations: The probability of the formula g is 
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where Pf(t) is the transition probability matrix of the Markov chain with alls ^/-states changed 
to absorbing states, for time interval i; and // is the diagonal matrix with entries 1 for /-states 
and for ^/-states. For example, 



P a (t) = cxp 
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Multiplying all these matrices as indicated in Formula (JXJ) produces the outcome: 

-^)V2k 0.000918 



M 1 (ff) = |e- 8 (e^ 
which is less than half the actual value 



3 First problem: final transition 

A problem arises with paths that enter state 4 during the time interval (3,4]. These paths are 
counted as non-satisfying by the algorithm (Formula ([1} only counts the paths that are in a c-state 
at time 4.), while they have passed through a c-state timely and actually may satisfy g. 

To solve this problem, P b \j c should be replaced by a matrix based on a Markov chain where 
additionally all c-states have been made absorbing, so that a path entering an c-state stays there 
until time 4. This is basically the same transformation as described by [2] for simple until formulas 
/i ^[<Ji,i>il h- Make all states except the /i A ^/2-states absorbing. In our example, we have to 
replace the factor P b y c (i) of Formula |T]) by: 



P b /\-. c {t) = exp 



(0 








°\ 


t 


-It 


t 

























<V 



2 








Figure 2: Modified example Markov chain for P a yb from Figure [T] 



and so, the calculated probability becomes 



//(<?) = (l,0,0,0)P o (l)I a P oV 6(2-l)/6n(3-2)J 6 nA^c(4-3)J c 
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= i e - 6 (l - e- 2 ){e^ - e~^)V2 ps 0.00293 (2) 



which unfortunately is still wrong. 



4 Second problem: intermediary transitions 

The remaining discrepancy after the first correction reveals another problem. In the example, 
there is a transition from 6-state 2 back to a-state 1. According to Formula ([T]), the path 1 *~S 
^ t-i.5^ ^ t ~ 1,8 > 2 *~ 3 ' 2 > 3 is counted as a path satisfying g, as it continuously satisfies a V b 
during the interval (1,2). However, the semantics requires that one choose when t\ g [1,2] has 
come. This must happen upon entering state 2 (a b A ^a-state) at the latest, so t\ = 1. After t\, 

t—l 5 

one is no longer allowed to enter a A -16-states, so the transition 2 1 is forbidden. 

4.1 Wrong correction 

We could try to correct this problem by deleting transitions from &-states to a-states in the 
example. This, however, gives rise to two new problems: 

1. The exit rate of state 2 (in the Markov chain of Figure [T]) would change. 

2. What about a A 6-states? These states are allowed both before and after t\. If t\ has passed, 
switching back and forth between a A 6-states and b A ^a-states should be allowed, while it 
should be counted as an error to enter an a A ^6-state. Before t\, the opposite condition 
holds. It is impossible to make a subset of the states absorbing in a way that satisfies both 
conditions. 

4.2 Better correction 

To solve the problem mentioned above, I propose to add extra states to the Markov chain for 
P a vb- Introduce a second copy s' of each a-state s that has a 6-predecessor state. One copy (s) 
stands for H\ (possibly) has not yet passed" and the second (s') for "ii has passed definitely". 
So, transitions from b A ^a-states to s are deflected to s' , and if s satisfies a A ->b, then s' is an 
error state and is rendered absorbing. 

In our example, we have to replace the factor P a \jb{t) of Formula © by the transition proba- 
bility matrix of the Markov chain shown in Figure[2j denoted by Pa V b(l) (I treat state 1' as fifth 



3 



state): 



f- 


-2t 


2t 








o\ 







-2t 


t 





t 


















































<v 



and so, the calculated probability becomes 



^(g) = (1, 0, 0, 0) P a {l)l' a P' ayb {2 - 1KA(3 - 2)I b P bA ^ c (A - 3)J C 
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which is the correct answer. 

In this formula, I also modified I' a and I b to include the transformation between the four- and 
hvc-state-Markov chains: 
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5 General formulation of the corrected semantics 

When we extend the above corrections to general until formulas, we get the following basis for an 
algorithm to compute the probability of until formulas: 

The probability of a formula of the form 

9 '■= fl ^[01,61] h U[ a2 ,b 2 ] f 3 ■ ■ ■ fn-l ^[a n _i,6 n _i] fn , 

where the intervals do not overlap, is given by 

» s {g) = n s ■ P h (ar) • I' fi ■ P' fiVf2 (h - ffll ) • 1% ■ P h (a 2 - h)- 
I' h -P' h v h (b 2 -a 2 )-I>} 3 -P h {a,-b 2 )-.. 
T U-2 ■ • P /„- 2 v/„_ 1 ( & «-2 - a„_ 2 ) • I'J^ ■ P / „_ 1 (a„_ 1 - 6„_ 2 )- 



■ Pfn-lA^f n (bn-l - On-l) ' If n 



(3) 



where Pf (t) is the transition probability matrix for time t corresponding to the Markov 
chain where all /-states are made absorbing; tt s is the starting probability distribution 
(which in our case has unity for state s and zeroes otherwise); ^/ 4V / 4 1 (t) is the transi- 
tion probability matrix for time t based on the extended Markov chain (details follow) ; 
and 7j. and I'f i+1 are the matrices that map /,-states and /j + i-states, respectively, to 
and from the extended Markov chain (details follow). 

The following table shows which transitions the extended Markov chain contains, i. e. the Markov 
chain to base P'j. y j. +i (t) upon. As mentioned above, the states are the same as the original Markov 
chain with an additional copy s' of each /j-state s that has a / i+ i-prcdccessor. 



4 



If the original Markov chain contains a transition s — > t, then the extended Markov chain 
contains the following transition(s): 





s' is a 
i' is added 


dded 

£' is not added 


s' is nc 
£' is 
added 


)t added 
t' is not 
added 


s 1= fi A -'fi+l 


s — > t; s' is absorbing 


s —> t 


s |= fi A fi+l 


s At and s' A £' 


s —yt and s' \ t 


s — > i 


s 1= "'fi A fi+l 


impossible 


s — > r 


s — > i 


s |= ~>fi A -'fi+l 


s is absorbing 



7y. and can be used to convert the probability vectors to and from the extended Markov 

chain. At time <2j, all probability mass should go into the first copy s of a state which has a second 



copy s , so 



1 if s = t and s satisfies /j 
otherwise 



Both copies of fi A fi+i -states are allowed at time 6, (the latest possible ij), so their probabilities 
should be added in the modified I'J '■ 



1 if s = t or s = t' and t satisfies fi+i 
otherwise 



5.1 Correctness 

To convince ourselves that the product in Formula ([3]) corresponds closely to the semantics given 
at the beginning of the article, let us look at some of its factors. 

• Pa n -i '■= Ifn-i ' ^ > /™-iA-./ ra (&n-i — fln-i) • If n ■ 1| can be seen as a vector of probabilities: 
P „_i( s ) is the probability that one gets a path for which there exists a t n -\ € (a„_i, &„_i] 
such that the path is in /„_i-states during the time interval [a„_i,i„_i) and it is in a /„- 
state at time t n , under the condition that it is in state s at time a„_i, as shown by [2]. (Note 
that ^ a„_i as the path has to be in a / n _i-state at time a n -\. This is not a relevant 
difference as the paths with t n -i — a„_i form a set that has probability 0.) 

• Let Mb n _ 2 := If n _ t ■ Pf n _ 1 (a n -i — b n -2) ■ If n -i- Then, M& n _ 2 (s, t) is the probability that a 
path is in / n _i-states during the time interval [6 n _2, a n -i] and ends in state t at time a n _i, 
under the condition that it is in state s at time 6„-2- 

Let Pb n _ 2 '■= Mb n _ 2 ■ P „_i< So, -P&„_ 2 ( s ) i s the probability that one gets a path for which 
there exists a t n -i G (a n -i, suc h that the path is in / n _i-states during the time interval 
[&n-2) ^n-i) and it is in a / n -state at time t n , under the condition that it is in state s at time 

K-2- 

• Let M an _ 2 := I' fn _ 2 ■ ^/ n _ 2 v/ n _ 1 (^n-2 - a n - 2 ) • ^X.^ Then, M a „_ 2 (s,i) is the probability 
that one gets a path for which there exists a i„_2 € (a„_ 2 ,6„-2] such that the path is in 
/n-2-states during the time interval [a„_ 2 , tn-2), it is in / n _i-states during the time interval 
[t n -2, b n -2\ and is in state t at time b n —2, under the condition that the path is in state s at 
time a„_2- 

Let Pa n -2 ■= M an _s-P bn _ 2 . (Note that •/ /n _ 1 = i}^.) So, P a „_ 2 (s) is the probability 
that one gets a path for which there exist i„_2 € (a n _2, 6 n _2] and £„_i G (a ra _i,6 n _i] such 
that the path is in / n _2-states during the time interval [a n -2,t n —2), it is in /„_i-states 
during the time interval [i„_2,t n -l) and it is in a /„-state at time t n , under the condition 
that it is in state s at time a„_2- 
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• etc. 



• Finally, let Pq :— Mq ■ P ai , and Po(s) is the probability that one gets a path for which 
there exist t% € (ai, 61], . . . , i n -l S (a n -i, &n-i] such that the path is in /i-states during the 
time interval [0, ti), it is in /2-states during the time interval [ii,^), . . . , it is in / n _i-states 
during the time interval [t n _2,in-i) and it is in a /„-state at time t n , under the condition 
that it is in state s at time 0. 

The first factor in the product, 7r s , serves to uncondition on the initial state. So, overall, the 
product 7r s • Pq calculates the probability that a path satisfies the semantics given. 

6 Concluding remarks 

The correction presented above provides a calculation principle to find the probability that a 
path satisfies an until formula, corresponding closely to the intended semantics as given in the 
introduction. It does no longer require to stay in /^-states overly long; it does no longer allow to 
switch back and forth between and /^-states too often. 

The present note passes over a choice that one has if intervals overlap: Would a E7[i 31 b 0" [2,41 c 
be satisfied by a path that jumped from an a A ^6-state to a c A ^&-state in the interval [2, 3], i. e. 
a path with t\ = t%l [T]'s remarks about overlapping intervals suggest they choose to forbid such 
paths, and my formulation of the semantics is aligned thereto; however, in some cases it may be 
more intuitive to allow them. It is possible to solve this discrepancy by adding the probability of 
a t/[2,3] c if desired. [5] choose the other way for simple until formulas: they consider /2-states as 
satisfying h U [0M] f 2 . 

The main goal of pQ was to prove decidability of CSL model checking. This erratum does 
not invalidate their proof idea; it only requires to fill in slightly different matrices in some proof 
parts, but the main argument - namely, that basic operations on (matrices containing) algebraic 
numbers produce (matrices containing) algebraic numbers again - remains valid for the modified 
matrices. So, it still holds up that CSL model checking is decidable. 

Acknowledgement. Most of the above matrix calculations have been performed using Maple. 
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